Graphical user interfaces (GUIs) have become very popular in providing users of computers, particularly users of computers with a mouse or other input pointer devices, with access to computer software, including menus, software options, desktop components, images and other graphical objects. WINDOWS XP®, by Microsoft Corporation, of Redmond Wash., and MAC OS X®, by Apple Computer, of Cupertino Calif., are popular computer operating system implementations that allow users to interact and manipulate desktop software (e.g., word processing, graphic design, presentation software and web browsers) using a variety of graphical components. Those knowledgeable in the art will understand that a graphical component may be displayed visually to users as at least one image displayed on a computer monitor.
In recent years, there has been a tremendous proliferation of computers connected to a global network known as the Internet. The Internet provides a transport mechanism through standardized communication protocols, such as the Hypertext Transport Protocol (HTTP), for communicating web content in the form of text, graphics, sounds, animations, video, and computer executable code.
A web browser is a client application or operating system utility that communicates with server computers via FTP, HTTP, and Gopher protocols. Examples of popular web browsers include INTERNET EXPLORER®, by Microsoft Corporation, of Redmond Wash., and FIREFOX™, by Mozilla Corporation, of Mountain View, Calif. Web browsers conventionally receive electronic documents from the network and present them to a user. In addition to being able to display text, web browsers are also typically able to display graphics and other multimedia content (e.g., video, audio, vector graphics, and vector graphic animations). Electronic documents may include text documents, conventional markup documents (e.g. HTML, XML, and others), graphics and other multimedia content, data files, and code that can be executed or interpreted by the web browser or other system components.
In addition to data and metadata, HTML documents can contain embedded software components containing program code that perform a wide variety of operations. These software components expand the interactive ability of an HTML document's user interface. FLASH® (plus ACTIONSCRIPT®) JAVASCRIPT™, and VBSCRIPT™ scripts are examples of software components that are embedded in HTML documents. A browser executes each script as it reaches the position in the script during interpretation of the HTML document. Scripts loaded during interpretation of the document may modify the document if the browser supports dynamic HTML (DHTML). Scripts may respond to user activity (pointer events or keyboard events), may post data to the server, and may request and receive data from the server. In recent years, GUIs have been implemented inside of web browsers using software embedded in electronic documents and transmitted from server to client over the Internet. Such mechanisms can, for example, use JAVASCRIPT and DHTML capabilities, or ACTIONSCRIPT and FLASH capabilities, and are well understood by those skilled in the art.
Techniques and popular conventions for managing graphical objects are well known by those skilled in the art of modern computer desktop usage. For example, in a windows-based environment such as that provided by WINDOWS XP, a desktop window object may be resized by clicking near the window border using a user input pointer, controlled by, for example, a mouse, and dragging the user input pointer to a different screen location in order to specify new window dimensions.
Until recently, the ability to manipulate and interact with graphically robust environments was limited to traditional desktop-based applications. As web browser applications become increasingly sophisticated, much of the functionality previously reserved for desktop-based applications is now being migrated to the browser-based applications. Using techniques widely known by those with normal skill in the art (for example, through asynchronous JAVASCRIPT and XML (AJAX), and DHTML), graphical-based desktop functionality can be programmed into web browser applications and delivered over the Internet. For example, Google, Incorporated, of Mountain View, Calif., offers browser-based word processing and spreadsheet applications with robust graphical user interfaces to allow users to create and edit documents on the Internet.
Techniques and conventions for interacting and manipulating graphical images in web browser programs often employ controls that are similar to those used by desktop software applications. Users often select graphical objects for manipulation, for example, by moving a user input pointer into the region of the object and clicking the input pointer to initiate the selection and manipulation process.
More than one graphical object may be displayed in a web browser at a given instance. These graphical objects may be positioned on the display such that one or more graphical objects partially or fully overlap another graphical object. In these instances, it is of particular interest how the web browser software discriminates, and the granularity at which the discrimination is made, between two or more overlapping graphical objects during the process of graphical object selection.
Prior art graphical object selection techniques in web browser programs often employ rectangular bounding boxes around the perimeter of graphical object images (for example, raster images in the JPEG, GIF, and bitmap formats, and vector images in the form of SVG, to name a few) to define the object selection granularity. These bounding boxes define the image selection region for their respective graphical objects, meaning a graphical object may be selected for manipulation when the user input device pointer is within, or clicks within, the perimeter of a bounding box.
Two-dimensional computer-aided design and graphic arts programs (“graphics applications”), such as VISIO® and MICROSOFT OFFICE (Word, Power Point), by Microsoft Corporation, of Redmond, Wash., and ADOBE ILLUSTRATOR, by Adobe Systems Incorporated, of San Jose, Calif., generally allow users to organize their designs and artwork into layers, more or less analogous to sheets of acetate bound together. Each object on a layer has a stacking order such that objects can be on top of one another. Similarly, layers are also stacked on top of one another to form the final work. The final work is rendered as a two dimensional design to be viewed on a video display, printed, etc. To the extent that one or more graphical objects are layered over other graphical objects, some graphical objects may be partially or fully occluded in the rendered two-dimensional design. Programs conventionally provide a mechanism to reorder object layering (e.g. via menu controls to “bring to front”, “send to back”, etc.). Some programs also provide a “layers palette”—a taxonomical tree of the layer identifiers—allowing the user to select the layer on which to draw, reorder the layers by dragging and dropping them with the mouse, and move art from one layer to another.
Controls and techniques associated with graphical object selection may employ a layering scheme when a plurality of objects is present on the same computer display. In situations where two or more graphical objects overlap on the screen, the rectangular bounding box selection granularity techniques used by prior art may make it difficult to select the desired graphical object.
Although raster images are in fact rectangular in nature (that is, they have height and width dimensions), the bounding-box selection regions used by prior art may overly approximate the actual useful selection region of a graphical object, since the visible region of the displayed object may be non-rectangular. Those of ordinary skill in the art of graphic design will understand that this is possible when a raster image contains both transparent and non-transparent pixels.
For example, a raster image of a circle may define the circle portion of the image using non-transparent pixels, while using transparent pixels to represent the remaining portions of the image within a rectangular bounding box. Because the selection region bounding box includes both the transparent and non-transparent pixels, it is possible to select the circle graphical object by selecting within the bounding box at a point containing transparent pixels (that is, at a point outside of the circle portion). It may be useful to limit the selection region of a graphical object to visible image areas represented by non-transparent pixels, such as circumstances when there are overlapping graphical objects displayed on the same user output screen. Prior art fails to provide a mechanism for supporting said limited selection capabilities.
Although techniques for object selection are well known to those skilled in the art, these methods of object selection may be difficult and frustrating when a user attempts to select an object that is not rectangular in shape, when conventional bounding box mechanisms are used. Frustration may also arise when a user attempts to select a single object from multiple objects displayed within an area of computer display that are in the state of object overlap or occlusion. These controls may also be non-intuitive and difficult to learn for inexperienced users.
There may also be considerable discovery involved with finding and learning how these controls function in a foreign or new environment, which may cause significant frustration for new users.